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The present invention irelsitiea to an input appaarap-ua 
for- generating control signals far a computer. 

5 Znteoducbion -to the Invention 

The tasks performed by the operator of a coniputer 
have defined the devices through which input data is 
generated. In the case of a personal computer^ a keyboard 
and a mouse are used. 

10 Design and operation o£ computer pexipherals 

maintains the difference between computexs and other 
equipment such as radio, hifi and television, The 
recjuiremenr of a Icsyboard and mouse i£s bec^oming 
increasingly perceived as a major barrier to the wider 

15 use of computers in a much broader range of activities • 

An example is the emergence of MP3 and other 
related audio c^ompression standards. Those provide high 
quality compresaion of audio data. Most radio stations 
ar-ound the world are now oble to broadcast over the 

20 internet, In additional to their tradirional location in 

the electromagnetic spectrum. Furthermore, it has become 
possible to store an entire CD collection on a low cost 
campuccr hard disk, Moy/ever, the computer, in its present 
form/ is not considered as a serious alternative to radio 

2S or hifi devices- A similar situation exists with video 
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data. The preferred viewing device, except when editing/ 
is a traditional r.ftJevij^ion set- 

Computers are increasingly capable of receiving and 
manipulating many different media types within a common, 
5 easily used, computer-generated environment. However, the 

method of supplying input to the computer terminal has 
restricted rne wider use of this technology. The keyboard 
and mouse are best operated at a deck, and this prevents 
compu-ters from being considered ^s replacements for a 

iQ broad range of conventional electronic equipment. As 

computer and internet technology develops, increasinqly 
the restrictions placed upon it are in rhe way the user 
interacts with computers through an input device. 

The digitisation of graphic design, video and film 

13 erditing has led to the development of improved devices 

for interaction with image data. The most widely used o£ 
input devices in this context is the graphics tablet, 
which is operated in the manner of a pencil-with-paper . 
On a large graphics tablet, it is possible to provide an 

20 area having the function of a keyboard, and this may be 

operated co genera Le ot^casional text where this required. 

In three-dimensional computer modelling, no single 
preferred peripheral device has emerged. Several systems 
are known, optimised for particular applications. An 

as example of this is radio tracking, which provides three 
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dimensions of position and three dimensions of rotation. 
In a virtual reality application, a radio receiver is 
fixed to a users head-mounted display, and the position 
information obtained by analysing data from a fixed 
S transmitter is used to determine stereoscopic images for 

the users eyes. The images are updated so as to provide 
an appropriate view for the angle and position of the 
user's head. Similar devices may be used to track the 
position at a hand, including devices thAn use ultrasound 

10 to determine orientation. Hand gestures aresulting frorn 

finger movemeut. may be tracked using a data glove- 
However, none of these devices is suitable for replacing 
a keyboard or mouse diie to the requirement to suspend the 
devices in space in order to generate position data in 

IS the third dimension. 

Graphics tablets and three dimensional input 
devices may be suitable for replacing the keyboard and 
mouse in certain aipplications . Howeverr their high cost, 
and other practical considerations, molcG rhem unsuitable 

20 replacements for the widespread, low cost and ubiquitous 

koyboard ond mouse of the personal compuT-er. 



eummmx^ oiT tlia Xnv«ntlon 

It is an aim of the present invention to provide an 
improved input apparatus for supplying control signal 



25 



s to 
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a compp-ter. 



• Figure 1 shows a sensor and a compurer terminal; 
5 Figures 2 and 3 detail construction of the aenaor 

shown in Figure 1, including microphones and a sensor 
core; 

Figure 4 details components of the sensor core 
shown in Figure 3, including analogue to digital 
10 converters, orientation sensors and a digital signal 

processor; 

Figure S details the digital signal processor shown 
in Figure 4 ; 

Figure 6 details one of the analogue to digital 
IS converters shown in Figure 4; 

Figure 7 details a frequency domain analysis of 
audio data recorded from one of the microphones shown in 
Figure 2, in response to a firsc cype of coucn event; 

Figure 8 details a time domain analysis of audio 
20 data recorded from two of the microphones shown in Figure 

2f in response to a second type of touch event; 

Figure 9 details steps executed on the digital 
signal processor shown in Figure A, including a step of 
identifying a drag position, a step of identifying a hit 
25 position and a step of identifying sphere orientation; 
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Figure 10 deT^aiXs the step of identifying a drag 
position GHown in Figure 9, including a seep of 
corrolacing with templates and a istep identifying 
inrersecting ares/ 
5 Figure 11 details templates tMat are used in the 

step of correlating with T:eniplateB shown in figure 10; 

Figure 12 illustrates the sxep or identitying 
intersecting arcs shown in Figure 10; 

Figuro 13 details the step of identifying a hit 

10 position shown in Figure 3; 

Figure 14 details the orientation eensora shown in 
Figure 4/ including a magnetic field sensor and a 
gravitational field sensor; 

Figures 15 and 16 detail the magnetic field sensor 
15 st^own in Figure 14; 

Figures 17 and 18 detail the gravitational field 

sensor shown in Figure 14; 

Figure 19 derails the step of identifying sphere 
orientation shown in Figure 9; 
20 Figures 20 and 21 4etail construction of a charger 

and receiver unit for use with the sensor and computer 
terminal shown in Figure 1; 

Figure 22 details components of the computer 
terminal shown in Figure 1, including a memory; 
25 Figure 23 details contents of the computer memory 
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Figure 2 4 sammarises steps performed by the 
computer rerminai shown in Figure 1, including cl step of 
calibrating the sensor and a step of using the sensor; 
5 Figure 2 5 derails the step of calibrating the 

sensor shown in Figure 24; 

Figure 2 6 details the step of using the sensor 
shown in Figure 24, including a step of processing touch 
event daKa; and 

10 Figure 21 details the step of processing touch 

event data shown in Figure 26, 

D«i^«ii.X0d. s«ac2^dLpt±c3zi of Tho Prararx-ad mn&Qdixnan'Ca 

A computer terminal is illustrated in Figtare 1- The 
iS compviter- ccsarmixial loi coTnpxlses a high resolution 

display panel 102 and stand^i'd personal computer 
cix-cuitx-y . The display 102 ic the only vipitolc paart of 
the computer. The components of the computer ay© biailfc 
in to the display housing. The cocnputer is connected to 
20 the internet, and provides access to media of many 

dirferent types, including audio^ video, applications 
Buch as wordproceasors , and so on- This highly varied 
ftiiactionalipy is provitSed by the combination of an 
iiatem&t barowsear eogtivas-e applioation and a cpraplnical 
25 user* interface enviironmetit such as X-Windows - This 

combination of technologies, both hardware an.d aof twa.re ^ 
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with information presented through a graphical display, 
may be concidered aa a computer generated environment- 

In order to interact with a computer generated 
environment of this type, a user must have convenient 
5 control ov-ear hho position and accivation of a cursor or 

poizit<ftr 103 , well an t:h.e €il:>ilit:y to encer cexc . These 

facilities are provided by a miilti -purpose input device 
104 having* the £oi?m o£ a. spli^are - THe oplicsare 104 is 
approximately aix cent irnet res iia diam^tear , wHi,ch ic 

la about Che aize of a tennis ball, in its simpla^c foarm of 

uBCr tbe apliere is lield in the left hand, while a finger 
of the right hand traces across its surface in order to 
affect the position ot the pointer 103 on the screen. 
The sensor 104 is configured such that it may be rotated 

IS in any degree by the left hand, while the position of 

the finger of the right hand on the sensor surface is 
considered only with respect to the display 102, In this 
waiy , Che sphere present o an infinite s-ur-face over which 
the pointer may be moved. 

2 0 While deliberate movements of a finger across che 

surface are interpreted as a movement of the pointer in 
the computer-gansrated environment . manipulations of nhc 
left hand, generate signals that are ignored. The tracing 
of a finger across the surface of the sensor 104 is a 

2S first method of generating input data. In traditional 

computer-generated environments, a click or double click 
may be used cp signify an action to be performed with 
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ireHp«et to tho location preaently oocupie^ toy Ciie 
pointer an bhe scaraen. i02. This is achieved by a cap or 
double tap on the sensor 104, regaarded as a hit. The 
position of the hit may be used to determine the nature 
5 of the command to be performed. 

The sphere may be used to enter text characters- To 
signify a change from the previously described method of 
operation, the sensor is hin, fairly hard, anywhere on 
its surface • Thereafter, movemehns of a finger on the 

10 surface may be interpreted as alphanumeric characrer^ - 

Tesct entry tnay b« performed one character at a time, 
using character recognition. A^lhemsit ively , for high 
speed text entry, a form of shorthand is recogn-ised. 

The sphere 104 has a slightly roughened surface, 

15 and this results in the generation of random noise as 

Che finger is dragged across iL- The noise 
characteristics are analysed in order to determine the 
pooicion of a drag evenc of this nype , An arrangement of 
sensar;3 to perform deteccion of noise generated in this 

3 0 way is shown in Figure 2 . The shell of the spliere 

comprises three millimetre thick silicone rubber, and 
this is covered by a fine nylon felc thac provides the 
noise-generating surface. Directly beneath the silicone 
rubber shell, arranged equidistant from each other, are 

2S four microphones 20i, 202, 203 and 204. These have a 

frequency range substantially equal to that of human 
hearing, and are extremely cheap. Each microphone may tie 
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conaxdereii as being at: Che cencre o£ a face o£ a three** 
sided pyramid, or regular tetrahedron. The angle made 
between any two microphonea at the centre of rhe sphere 
is 2arctan<sqrt (2) ) ; or approximately 109,5 degrees. 
5 Signals received by an individual micarophone may be 

analysed to identify the proximity of a noise-generating 
J djrag event , and the results of an analysis o£ this typ«« 

from two or more microphones are combined to identify 
Che locacion of cJie event on the surface of the sphere. 

10 Puxtlier details of the consceruct ion of the sphere 

104 are ehown in Figure 3, During manufacture, the 
silicone rubber shell Lg created in two halves 3 01 and 
302. A centsral cox-e 303 contains the ciarcuitry of the 
aenaor. Between the core 303 and the silicone arubber 

15 shell is a layer of acoustically isolating and shock 

absorbing polyester fibre 304. This construction firstly 
ensures that microphones 201 to 204 are sufficiently 
acoustically isolated from each other, and that each 
micxopiione only receives sound from the silicone rubber 

20 shell- As a significant further advantage, however, this 

construction provides a very high level of shock- 
imtnuxxity, &o that the aphcrc 104 may be Kandled 
extremely roughly, may he dropped, thrown, or otherwise 
subjected to axcreme aceeler-ation forces without any 

25 damage whatsoever. The microphones 201 to 204 are 

embedded in homogeneously moulded silicone rubber 
mountings in the shell itself, and so are also extremely 
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arobus^ i.n tr.his respecr . 

Tbe c\nro halves 3 01 and 3 02 o£ che apliere 104 are 
combined using an acoujstlcally hOTTlogeaous silicone 
ar\ibb*a3r eeal » The cor-e 3 03 contains a recliargeaiale power 
5 ao\ir-<=e* and tKia must areceivo powcx excema-lly wlieu 

reeharsring ia required. In ax-der to avoid compc-omiBing- 
the structural integrity of the suxfaee by a wire 
connection, an inductive loop 3 05 provides accents to a 
recharging current source . 

The circuitry contained wi.thin the central care 3 03 
shown in Figure 3 is detailed in figure 4 . A first 
analogue ro digital converter (A-D) 4 01 receives 
analogue audio signals from the first two microphones 
2 01 and 2 02 a second analogue to Cigital converter 4 02 

IS receives analogue audio signals from the second two 

micK-ophones 203 and 2 04 . Bach converter 4 01 and 402 has 
stereo channels, that are converted into a multiplexed 
digital signal that is supplied over commpn connections 
to a digital signal procR«Kor (DSPJ 403 - The digital 

20 signal processor is a Motorola DSP5SS03, that includes 

highly optimised arithnietic and storage circuits 
designed for the analysis and processing of audio 
signals. Data for the PSP56603 is available from 
http://ebus.mot-sps .com, 

25 Analysis of signals supplied to converters 401 and 

402 results in position signals being generated, and 
these are transmitted digitally from a transmitter 404 
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tQ a recei>fey connected fco the computer cerrnln^l 101. 
Preferably^ poeitional aignalo are ge=neraced with 
reBpect to the tarmixial loi, irrespective o'f any degree 
of rotation of the sensor 104. This is facilitated by 
5 orientation sensors 40S, The orienLatxon sensors 

characterise sensor orientation as a first rotation RM 
about a horizontal axis due to the Earth's magnetic 
field and a second rotation RG about a vertical axis due 
to the Earth's gravitational field. By combining these 

4.0 data with data about the positions Of toucn events 

occuring on the surface of the sphere, it la possi3Dle to 
intearpret ueear gestures on the sphere with refex-ence to 
a standard space in which the computer terminal 101 ia 
located- Thus, for example, a forwaard dragging movement 

15 of the finger towards the screen lox will move the 

cursor upwards, regardless of the orientarion of the 
sphere • 

The core includes a power manager circuit 4 06, that 
provides Xovt power and shutdown modes for the DSP 403 

20 and other circuitry- The power manager 4 06 receives 

power from a rechargeable NiMH battery 4 07/ and also 
facilitates rectification and current regulation of 
recharging power supplied from the inductive loop 305, 

The digital signal processor 4 03 shown in Figure 4 

2S is detailed in Figure 5 ► Several parallel data and 

address busses are present within the DSP, and these are 
summarised for the sake of clarity by a simplified 
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wiring connection 501. Timers 502 provide pailse width 
timing capabilities that are used to provide a 
xneasuremexit of signals from the orientation sensors 40S. 
inpxit MdL aucpuc. circuits (I/O) 50 3 provide several 
5 conxxections, including those us«d. for the A-D convercers 

401 and 40a. The pxogvam ROM and E^AM S04 includeia 
bootstrap and ciontarol instructions Coar cso-ordinaiiiag the 
interfaces with cihe other circuitry in the core 303, and 
ror performing signal analysis. TUi X data RAM and a Y 
xo 5iata RAM SOS and 506 provide a pair or operands per 

insttx-uction cycle to an ax-ithmetic and logic unit (J%JjU) 

507. The ALU is thereby capable of fetching and 
mulciplying cwo data operands from X ajaa Y memorv SOS 
and 506 in every instruction cycle. This arrangement 

15 racllicates efficient imp 1 emendation of the processing 

algorinnms that: are required in order no determine the 
position of touch events on the surface of the sphere. 

Eacli of the D-A circuits 401 and 402 comprise 
circiaitry- as shovm In Figure s. A fiarst channel pr^e- 

20 amplifier SOl receives an unamplified audio signal from 

a microphone and increases its intensity to that which 
is suitable for digital to analogue conversion. An anti- 
alias filter 602 removes frequency components above half 
the sampling rate, ao as to exiaure that sulDSequent 

25 frequency analysis of audio data gives an accurate 

representation of the epoGtrum . The output farom filcear 
602 is supplied to the left input of a stereo sixteen 
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bit low power analo^e to digital converter chip 603. 
The aampli-ng arsifco is 44-lkHs, aad ixxtegx-atcd circuits of 
thia type are widely availaible at low cost. Another 
channel is implemented for a second microphone using 
pre -amplifier 604 and aati-alias filter 605. A common 
multiplexed output is supplied from the A-D converter 
chip to the DS£» 403, Additional clock and word 
synchronifisation signals aare emitted £oir claarity. 

An analysis of several seconds of audio daca from a 
ainsle channel is shown in Figure 7, The vertical axis 
represents time, and the vertical axis represents 
frequency. In thia graph, the amplitude of a particular 
frequency component at a particular time is represented 
by density. The graph shows a plot of a signal that 
results fx-om the movement of a finger acroep the surface 
of the sphere . 

At the start of the plot 7Qi the fingertip is 
distant from the microphone. Sound waves reaching the 
microphone aare primarily transverse wave a , oscillating 
perpendicular to their direction of propagatiion . The 
silicone rubber filters high frequencies tout h^s little 
effect on the low f rBt5uencies . Thus, the signal reaching 
the microphone, regardless of its actual amplitude, 
contains an inaication of the distance due to the 
relative strengths of high and low frequencies. As the 
finger tip moves aloser, higher frequencies increase, 
while the strength of the low ones remains substantially 
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cHo same. lia fiiigerci-p becomes incsreasing-ly close To 

fcK© Tn±c3rophonc , eit 703, a completely new set of 
firequencxos Lb ^dded in the spect^xum. Ttils d.s due ro 
longitudinal waves being transferred across the 
5 thiclciiesa of the silicone rubbear from the fiiagertipi 

directly to the microphone. As the microphone is 
approached, there is a mixture of both longitudinal and 
tran-averse waves, as identified from the two distinct 
area.c3 o£ nhe graph ac 703 and 702 . 

a.0 A final exceptional condition is reached when the 

fingertip ia directly over the microphone. The high 
frequency aomponcnto arc generated by i_he friction 
between the edge of the finger and the r-o\ighcned siortacre 
of the sphere- However^ when the finger is diarectly over 

15 the microphone, cheae high frequency components az-a 

masked, and the sound picked up by the microphone comes 
from the centre of an area of the fingertip alone. At 
704^ a sudden loss of high frequencies occurs because 
these frequencies ^re damped by the area of the 

ao fingertip. The low frequencies exhiJDlt a characteristic 

change, having a more balanced profile. 

These changes provide a chaaracteristic set Qf 
descriptions for finger drsigging events that occur on 
the surface of the sphere, py comparing the outputs two 

25 or three channels at once, the position -of a nioving 

fingertip anywhere on the surface of the sphere 104 may 
be identified. 
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Hitting the sphere, to perform the equivalent of a 
mouse click, does not contain as much frequency data, 
and 00 a differenc type of analysis is used. The speed 
of transverse sound waves in the silicone rubber shell 
s is in th« ox-der of only cwenty. mecres per second. This 

makes it possible to discern a time difference for 
wavefronta arriving at different mier-ophones . A pair of 
graphs resulting from a simultaneously digitised hit 
event are shown in Figure fi- Trace SOI is foT- the more 

10 distant microphone, and it can^- Jae seen chat this 

comrnences a short period after the second trace ao2« The 
difference in initial characceristiic waveCronts is in 
the order o£ two thousondchs o£ a second- This provides 
a reasonably accurate source of position data.. The 

IS traces 8 01 and S02 alec exhibit diff eranaes ii^ frequency 

content, which may be observed in the jaggednesae of the 
second trace a 03, which is the microphone nearest to the 
hit event- Waveforiri flOl reaches a higher peak, due to 
the lack . of damping provided by the finger over the 

20 point of impact jusc after the hit event has occurred. 

Several such characteristics may be analysed and the 
results coTnbincd sa as to identify a touch event 
characteristic to an increased level of accuracy. 

The main sequence a£ steps performed by the DSP 403 

25 shown in Figure 4 is sun^marised in the flow chart in 

Figure 9 - At step 901 a frequency domain analysis is 
performed on each of the four channels of buffered audio 
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data. AC stap 902 a question is aeked as to wheciiex- a 
drag or a hit event has been observed in the data. It is 
posaihle that neither is identified, either due to lack 
of ocourrenee, or because a clear charactearistic c^a^uioc 
5 be identified, I£ neither drag nor hit is present in the 

audio data, control is directed to step 905. If a hit is 
observed, control is directed to step 904, or, if a drag 
is observedl, control is- directed to step 903 . At step 
903 the drag position is identified. At step 904 the hit 
10 position is identified. 

Ac step 90 5 th-e sphere orientation is identified by 
analysing" daca from tbe axiennanion sensors 40S • A.t step 
906 a question is asJced as to whether any data needs to 
be transTnitted to the computer- For e^mple, if no toucb 
15 event has occurred, and the orientation has not changed, 

no data need be transmitted, thus saving battery life. 
If no events occur over a prolonged period of tiine, say 
twenty seconds, the sensor can be placed in a power down 
tnode- wnen held in elcher nana, even if not being used, 
zo tlic sensor will sense small changes in orientation, 

indicating tliac it is probably abouc to be used.. If data 
is available for transmission, control is directed to 
3t;ep 907, where data is cransf erred aver a serial link 
from the DSP 403 to tnc radio transmitter 404, for 
2S tranffmic«ian to che computer 101- 

The step of identifying the drag position 903, 
shown in Figure 9, is detailed in Figure 10- At step 
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lOQl the four chaiuiels of buffered aiodlo data are 
analysed to identify the three with the gareatea^ 
atnplibude. These three are the channels whose azialysis 
will yield the most accurate touch event 
characterisation. The three loudest channels are called 
A| B ajad C. AC .step 1002 the first of these three 
ehoxinels Is aelecced* At ?tep 1003 a frequency domain 
analysis ia perfoxroed, and the results of this are 
txomu&Hsed, such thafi thfi Icudesr freq-uezxcy component 
tias an amplitude of one. At seep X003 a correlation is 
performed with respect to a set- of templacea. Eacli 
template characterises a particular frec^ency irasT^ans^ 
that ia expected co occur at a known diatance from a 
microphone. Thus, with reference to Pigixre l , a template 
exists for the frequency characteristic at 701, 702, 703 
and 704. Each template has a different shape. The degree 
ta which actual microphone flata matches one of these 
templates indicates its proximity to the characteristic 
dis'cance of th^t template . 

A correlation score is geoearatcd as a sreaxilt of 
step 1004, and at step 1005 th.e two best scoring 
templates are selected. It is then known that the actual 
distanea of cine event from che miaroptiaiie for than 
channel is between the characteristic distances of chese 
two templates. At step 1006 tlie actual distance is 
identified by interpolating between the two 
characteristic distances, in proportion to the 
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difference between che template scares. This Identifies 
th.e eUaracteriatic distance for che channel, which may 
be DA I DB or DC I depending on which channel is being 
analysed. AC scep X0Q7 a question ia asked as Co whether 
5 there ip another channel remaining to be euialysed. If 

so« eonnrol is di.z-ected back CO Step 1002. 
Alternatively, each distance DA, DB and DC will have 
been identified- Ac step 10 oe incerse^cin-S' arcs ai-e 
identified across the surface of the sphere for each 

;.o characteristic distance, and ac seep xoos a point, P, is 

identified thac is defined by che nearest point at 
convergence for the three arcs defined by DA, DB and DC. 

The ceniplatBs chat are used in step 1004 in Figure 
iO are illustraced in Figure li- Template 1101 

x£ corpe^ponds to on ideal frec^ency response at a distance 

o£ D-4 0 Ttim away from a micx-ophonc • The signal at 701 in 
Figure 7 would closely match this template. Template 
1102 has a characteristic dis-cance u-2b mm, and roughly 
corresponds to the plot at position 702 in Figure 7. 

20 Templacie 1103 has a characteristic distance D»10 mm^ and 

would provide a high score for a signal occurring just 
after point 7 03 in Figure 7- Template 1104 corresponds 
to the fingeartip being directly over che microphone, and 
corresponds to point 704 in Figure 7. By selecting the 

25 best two corresponding templates, the actual distance o£ 

Che event from the microphone may be identified by 
interpolation between characteristic distances of the 
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templates . 

The distance of an event may be coasideareti as a 
notiouQl seance, aa frequency characneriscic^ may 
change Eos: di.£ferent: £inger sizes, applied pressure and 
5 onlie? 'v:a3riable £ae<oxrs . WhaCevcr t^he d5 seances are, DA, 

DB and DC dd£i.ne hhree e}ia3rac!tex-3.sbi.c sixaa , whose Ideal 
convergence point P is illustrated in rx^^ct 12 . The 
ideal convergence point is the same regardless o£ these 
variahle f accors • 

XQ xdencif Icaclon oC a hit position, shown at step 904 

in Figusrc ^, is detailed in Figure* 13, An step 1,301 the 
three loudest channels A, B and C axe identified. AC 
step 1302 the £irst of these channels is selected for 
analysis. At step 1303 the siignal is filteared. The 

15 filter removes frequencies below 2S0 Hz, as this results 

in a better analysis being performed. Preferably an FIR 
linear phase filter is used. However, an lllL filter, 
auch as that uged to generate the trace shown in Figure 
a, is acceptable^ with a slightly reduced accinracy of 

20 results. At ctcp 1304 an event start tlTTie Is identified 

by analysing the channel data. Ac step 13 05 a question 
is asJcfed as tio whether another charuiel r-emaine to be 
analysed, and if so, control is diraecad JaacD^ co step 
1302. Altemacively , scax-t times will have h©€Ma 

25 identified for each of channels A, B and C, and control 

is directed to step 13 06- 
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The difference between start times for a. pair of 
chanxiels identifies a distance from the cnid point 
between two microphonea - On clie eurface of tlie sphere, 
thia TTiid point ie expressed as a line. At step 13 06 

5 distances are identified for each combination of scarl: 

times, reaulcing in three lines, oi- arcs, being- 
identified across tK* s-ur-faeft o£ £b« spHeare . At step 
13 07 a characteristic common point is identified, in a 
similar manner to that shown in Figure 12. In theory, 

Lo two such arcs are required. However, three are used to 

improve accuracy. Four may be used, if the channel data 
for all four microphones is of sufficiently high 
quality. 

Orient scion of che eensor is defineca toy rotations 
IS RM and RG about Earth's magnetic and gravitational 

fields. Detail of the orientation sensors 40S shown in 
Figure 4 is shown in Figure 14, A magnetic field sensor 
1401 and a gravity field sensor 14 02 generate digital 
oscillation signals whose periods aara measured in order 
20 to ascertain orientations within respective fields. A 

multiplexer and counter circuit 14 03 provides 
interfacing and control signals for the oscillating 
circuits, and divides the frequencies down by an amount 
sultaJDle for highly accurate measurement by the timers 
25 502 in tne DSP 403 . 

The magnetic tield eensor 1401 is detailed in 
Figures x5 and IS, Figure 15 shows three mutually 
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o;c-tiiog*onal xnducnorB • Each inductor is less than ten 
millimetreo long. Figure 16 details a circuit: suitable 
for detecting the polarity and magnitude of, the Earth's 
magnetic fiftld with respect to each of the three 
induct oars . A logic gate Id 01 provides a positive ox- 
negative DC bias via resiscor 1602 to the presently 
selected inductor ISOI to 1503. Salection of an indu«=tox- 
is provided by logical control signals supplied to tri- 
etate logic buffers 1602, 1603 and 1604, An operational 
amplifier 1605 provides amplification for sustaining 
oBCillations whose frequency is deuexTnined by the 
inductance of the selected inductor 1501 to 1S03- The DC 
bias provided by r-esistor 16 02 drives the core of the 
selected indvictor to near aaturation, 

Near to saturation, a coil's inductance changes in 
r-aeponse no the applied field, even though the coil » s 
windings and core are fixed. The additional offset 
towards or away from saturation, resulting froTti the 
Earth's magnetic field, may be detecned in this way. By 
switching polarity of the DC bias in the coil, it is 
possible to determine the polarity of the Earth's 
magnetic field when different resulting oscillation 
freciuencies are compa.red. If there is no difference, 
this indicates that the coil is aligned orthogonally to 
thes Earth's magnetic field. The output from the 
operational amplifier 1605 is supplied as a logic signal 
to the counter 1403, and the DSP 403 determines the 
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precise frequency of oscillation for each of the coi3.B, 
in each polarity i arid thereby the alignraenc^ in tHaree 
dlmenaions, of magnetic North. The circuit requires a 
couple af milliamps of current to operare, and che 
s aenoors are extremely qmall and of low cost. A suitable 

induotoar^ of the type olicrwii in Figure 15, is Che SEK-M 
magneto- ind\ictive Benaor, available fx-am Precision 
Navigation o£ Menlo Par-lc, California. Uee of partly 
DaturatinQ indu-ctors Co detect tJie Earth's magxitttic 

10 field in this way ia detailed in US patent 4,851,775. 

The gravity field sensor 14 02, shown in Figure 14, 
is detailed in Figures 17 and 18 . An enclosed spherical 
container is half filled with a liquid having a 
substantially dirrerent relative permeability to that of 

IS free space, at a frequency of around 500 kHz. A suitable 

ligu.id is mercTj.ry, which has a relative permeability of 
around 0,7 at this frequency. Three coils 1702, 1703 and 
1704 are wound in close proximity to the container 17 01 < 
and are mutually orthogonal. Each coil is cormected in 

20 an oscillator circuit as shown in Figure la . Each coil 

forms the inductive part of a tuned circuit, that also 
comprises two capacitors. A logical HCMOS NOR gate 1801 
to 1803 provides amplification and a selection input to 
activate the oscillator circuits separately. The outputs 

25 from the oscillators are combined in a three-ir^put NOR 

gate 1804, so that an Inductor selected for oscillation 
by a logical high input to gate 1801, 1802 or 1503, has 
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ica eHaract eristic frequency pffeeente^i as a. aq-uax-e wave 
at the output of gate 1S04- The output from gate 1804 ie 
supplied to the counter circuit 1403. The frequencies of 
oscillation of the three coils depend upon the amount of 
immediately ad j acent mercury - The three frequencies are 
measured in the DSP 403, and interpolated look-up tables 
are used to determine the actual orientation of the 
eensoar with respect to the Earth' s gravicational f ield- 
Tlie circuit of Pigxire 18 requires less nhan one milliamp 
to operate , 

The thrae -dimensional vectors for magnetic and 
gravitational ambient fields may be represent ad aa two 
rotations of the sphere^ RM and RQ, about tbe magnetic 
North axis, and about the vertical gravitational axis. 

The process of identifying RM and RG combines 
measurements from botli the gravitational and magnetic 
field sensors. It is possible for either to experience 
interference, and the gravitational £ielc3 sensor 1402 
Tnay eseperienee instability due to motion of the mercury 
in the container 1701, particularly if the eenaor is 
moved ^bout rapidly, The step of identifying the 
orientation of the sphere 905 , ehovm in Figure 9 , is 
detailed in Figure IS. Ac step 1901 the rotation about 
magnetic North, RM, is identified. At step 1902 Kalman 
filtering is applied to the value of RM. A Kalman filter 
determines a measure of confidence in the current 
measurement, and increases low pass filtering when this 
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eonfidona© value ±m low. At cst-ep 1303 the rotacion RG o£ 
th© sphere abouc tlie vertical axis due ta graviny is 
determined, and at step 1904 Kalman filtcriixof is also 
applied to this value. At step 19 05, filtered vtalucs for 
5 RM and RG are stored for later transmission to the 

computer 101 when neaeosary- 

After an extended period of use. the batcery 407 
requires recharging. In order to recharge, the 
inductive loop 3 05 must be placed in an adjacent 

10 posicion to another matching inductive loop chac is 

supplied toy a ikHz power source, mis may be located in 
a charging unit as illustxaceca in figure 20, The outside 
of the sphere has a mark directly opposite the location 
of the inductive loop 3 OS, and this marJ^ must be 

15 uppermost when the sphere is placed on the recharging 

unit 2 0 OX- The recharger 20 0x may also conveniently 
double as the receiver for the data from the transmitter 
circuit 404, and a serial connection 2002 provides the 
connection for this data to the computer terminal lox. 

20 Details of the charger and receiver unit 2001 are 

shown in Figure 21. An inductive loop and oscillator 
2101 supply an alternating magnetic field to the 
mauctive loop 3 05 in the sensor 104 during recharging- 
During use, the sensor 104 transmits radio signals to a 

25 ra<aio receiver 2102. A central processing unit (CPU) 

2103 provides error correction of data received pver the 
radio link. A universal serial bus (USB) interface 2104 
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pxrovxcieB a connection CO the computer terminal 101 via 
the eex-ial ceilsle 2QQ2 . 

Tbe comp-uter texTniTial 101 ahowi in Figure 1 is 
decailaa in Piguare 2 . A central processing unit (CPU) 
S 22 03 pxrovid-ea co-oardiaafcion and prooessiug £or ttie 

terminal 101. Instructions and data for Che CPU 2203 are 
stored in mai.in mBmary 2204/ and a hard disk storage unit 
220£ facilitates non-volatile storage o£ data and 
several software applications - A modem 22 Q 6 provides a 

10 connection to the internet. A universal serial bus (USS) 

interface 2207 facilitates connection to the charger and 
receiver unit 2001. Touch event and orientation data are 
received from the sphere 104 via the USB interface 2207. 
A graphical processor 2208 provides dedicated graphics 

15 rendering capajDilitiea to speed up the display of high 

resolution graphical images on the display 102. An audio 
processor 2209 supplies audio signals to loudspeakers in 
the computer teiminal 101, and receives audio Signals 
from a microphone 2211 - 

20 The contents of the main memory 22 0-1 shown in 

Figure 22 are detailed in Figure 23. An operating system 
provides common fianctionality for software applications 
2302. A device driver 2303 for the sensor 104 is also 
stored in main memory 2204 while the computer terminal 

2S 101 is switched on. The sequence of operations necessary 

to operate the sensor 104 is detailed in Figure 24 . At 
ptep 24 01 the sensor 104 is charged using the charger 
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and receiver unic 2001 shown in Pignaress 20 ana 2x . At 
step 24 02 the sensor is calibrated. In order to ujss the 
senaori it is neceaoary to atore orientation data so 
that the device <irlver 2303 is able to determine which 
way is forward, bacJcwards, left and right with respect 
to the Earth's magnetic field, and therefore also 
suhstantially with rcapect to the terminal l-Ol, 

It La aoevimea herein that orientation of the sensor 
wichin the Earth's magnetic and gravitational fields 
effectively provides orientation ,with respect to the 
terminal lOl . Clearly this would be untrue if the user 
operated the sensor from behind the cerminal , However, 
for the purposes of practically operating Lhe sensor, it 
may be assumed that touch gestures on the surface of the 
sphere 104 that are made with respect to the Earth's 
maenemc and gravitational fields are also made with 
respect to the location of the computer terminal 101. If 
the terminal position is changed substantially, ic will 
be neoessary to perform the calibration an step 2402 
again. At srep 2403 the sensor 104 is used, and at step 
2404 the sensor 104 is recharged. The design of the 
charger and receiver unit 2001 is such that: the censor 
104 may be conveniently left at rest on the charger 2001 
whenever it is not in use. 

The step of calibrating the sensor 2402, shown in 
Figure 24, is detailed in Figure 25. At step 2S01 the 
computer requests the user to drag their finger from the 
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back CO the front of the sensor, moving- over the micidLle 
o£ the top . Af rer this is done , the compucer requests 
the uc3cr to cirag from left to right in the same fashion. 
Although both are not strictly neceataary/ thi? reduces 
the enror in aalibaratioa- At step 3S02 the orientation 
data from the sphere 104 is analysed in order to 
determine a user orientation angle AG/ about the 
vertical axis of the Earth's gravitational field. 

The step of using the senaor 2403, shown in Figure 
24, IS detailed in Figure 26. At fetep 2601 touch event 
and or orientation data is received froni the seneor 104. 
This includes the angles of rotation RM and RG about the 
gg^y^l^ig Tnagnetic and gravitational fields- At step 2602 
the user orientauiou angie AO calculated at step 2302 in 
Figure 2 5 is subtracted from the sphere rotation angle 
RQ in order to obtain a rotation value SG. At step 2 603 
a i^estion is aslted as to whether a touch event has been 
received, If not, control is directed to step 2606. 
Alternatively, control ie directed za step 2604. At step 
2 604 the sphere co-ordinatea of the touch event are 
rotated in opposite and equal degree co SG and RM, 
resulting in touch event data that has a stationary co- 
ordinate system, irrespective of the orientation of the 
sphere- At step 2605 the resulting touch event data is 
processed. Finally, at step 2606 the tearminal -oriented 
data is supplied to the operating system S3 0l vi^i the 
device driver 2303, for use by applications 2302. 
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The step of processing touch event data. 2 605/ Ghown 
in Figure 26, ia detailed in Figure 21. Ac step 2701 a 
question is asHed as to whether a large hit has been 
ret=eived. A la.rge hit is one where audio daca from all 

5 four microphone channels i3 extremely loud, indicating 

the user has hit the censor quite hard, request a change 
o£ mode. I£ a large hit is received, control is directed 
to step 2702 wheare the cu-arrent mode io swapped. 
Alternatively, if a large hit is not id&nt:i£ied at srep 

10 2701, control is directed to step 2703. Ac- step 2703 a 

question is asked as to whether the currently selected 
mode for the sensor is graphics mode or text: mode. If 
text mode is selected, control is directed to step 2706, 
Alcematively, graphics mode is identified, and control 

15 is directed to step 2704. At step 2704 any sinall hits 

are interpreted as tlie equivalent of mouse hutton 
clicks p Finger drag events are used to modify the X and 
V co-ordinates of the cursor. At step 2V05 the cursor 
position is updated - 

20 When used in text mode, control is directed from 

step 2703 to step 2706, At step 2706 any small hits are 
interpreted as the equivalent of GAPS, CTRl. and SHIFT 
events on a conventional keyboard. At tstep 2707 touch 
^vements on the surface of the sphere are interpreted 

as as character entry events. Alternatively, for high speed 

text entry, a form of shorthand can be used. 
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In an alcernotive embodimenr , the DSP 403 performs 
data compreasion of audio signals from the microphones 
201 to 204. The cQmjpressed audio data is combined wicli 
orientation data, and is transmitted co the computer 
terminal 101 for analysis to determine surface event 
characteristics . 

In the embodiments described above, the siensor is a 
paasive device, reqruiring a eound to be made by a touch 
event on the surface of the sphere. In an alternative 
emtJOdlment , an active sensor is provided. Sound may be 
injccced into Che surface of the sphere, and a surface 
presssure map may be constructed from sound 
characteristics that result from interference and 
reflection* If the sensor shell is made of a hard 
15 material, ultrasound may be used to generate a highly 

detailed pressure map. The pressure data may be usod to 
facilitate additional methods of data entry. An 
alternative method of detecting pressure is the use of 
multiplexed pressure sensitive electrical sensors, whose 
20 conductivity changes in accordance winh the applied 

pressure . 

The spherical shape of the sensor facilitates 
rotation in any degree without this making any 
difference to the appearance or feel presented to Che 
25 user. Touch events on the surface are made with respect 

to Che computer display 102, This has the psychological 
effect: of extending the computer generated environment 
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out into the space between the user and tht terminal. 

Although in the preferred embodiment the entire 
surface of the sphere is touch sensitive, it is possible 
that, for the purpose of providing a direct electrical 
5 connection during rechargiT^g, that a different 

embodiment, may include an insignificant portion of its 
surface wheare touch senaing io not f\ally px-ovided , Also, 
in certain embodimemta , the spherical shape may be 
distorted, as a result of squeezing or 4ue to a 

10 preferred distorted shape. However, a distorted 

spherical sensor of this type, having a suhatantially 
entirely touch position responsive smoothly curved and 
fully enclosed surface, are still considered a form of 
the present invention, 

15 The sphere, being the simplest of three dimensional 

shapes, provides a auitatole shape for universal abject 
mapping. A cornplex shape, such as a telephone handset, 
may have its surface mapped to the surface of the 
spherical sensor X04, and interaction with the handset, 

20 for escample to dial a number, may be effected via 

interactions with the sphere. Alternatively, the shape 
of a three-dimensional object mapped in such a way may 
be modified using the touch events on the sensor's 
surface. The shape, then changed, is remapped to the 

25 surface of the sphere, so that additional changes may be 

made- The universal shape of the sphere lends itself to 
interaction With a rich variety of complex shapes and 



Received 06-12-99 15:^5 from- 



To-THE PATENT OFFICE 



Page 34 



32 



. The orientation dAta o£ the sphere, although 
intended primarily as a means to eliminate ophoxe 
oriientation from gestures performed with respect to a 
5 coTi^Jucer teanninal, provides a useful means for 

navigating a three-dimensional computex-geaerated 
i ) environment . The orientation data may be used to define 

angles and trajectories in a fly-through of an 
artificial world. 
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1, A eapiputear input appaaratu-e Havincy tHe foxm of 
a f!iilly enclosed sphere, including 

toueli sensing m«ane foar geaearafeing position eigiaaXs 
indicative of the position of touch events occurring 
anywhere on the surface of aaid spheare, and 

transmiccing means arranged to transmit signals to 
a computer in response co said position signal^. 



2 - Appa3raGU3 according to 
ineludi.zig oiri ^ntiation aexialxig means. 



claim 1 1 



further 



3 . Apparatus according to claim a , wherein said 
15 orientation sensing means includes an ambient: magnetic 

field sensor. 



20 



4, Apparatus according to claim 3, wherein said 
magnetic field sensor comprises three mutually 
orthogonal magnecic field detectors. 



25 



5, Appciracus according to claim 2, wh.erein said 
orientation sensing means includes gravitational sensing 
means . 

6. Apparatus according to claim 5, wherein said 
gravitational sensi ng m eans comprises 
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a mobile inductor core in an enclosure, and 
three mutually orthogonal inductors responsive to 
the position of eaid mobile core. 

7. Apparatus according to claim 6, wherein a said 
ind>ictor ie included in an oscillating circuit. 

e. Apparafcuo accordii»g to claiii. 7, including 
counting means for measuring a frequency generated by 
said oscillating circuit 

■ 9. Apparatus according claim 6, including 

means for measuring an inductance of a said 

inductor . 

processing means for processing said measurement, 
wherein 

said processing means includes look-up means for 
identifying the orientation of said sphere with respect 
to gravity. 

10. Apparatus according to claim i. wherein said 
sensing means comprises a plurality of sound transducer 
means . 

11. Apparatus according to claim lO, wherein said 
sphere has ^ .uxface thac generate:. :.ound in «sponse to 
a touch event . 
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12. Apparatus according to claim 11, wherein said 
aai^d generaclng surface generates a noise -like sound in 
x-esponae to a dragging tnocion of a finger. 

13. Appaaracus accoirding to claim 11 ^ wherein said 
sensing means comprioeB a plurality of microphones. 

14. Apparatus according to claim 13, inclu.ding 
processing means arranged to compare signals from said 
microphones in order to identify a characteristic of a 
touch event, 

15. Apparatus according to claim 14, wherein said 
characteristic is a position. 



IS. Input apparatus for a computer with a graphic 

display means, liavlng the form of a fully enclosed 

sphare , inol\ading 
20 couch sensing meanff for generating position oignal3 

indicative of the position of touch events occurring 

anywhere on said spherical surface, 

transmitting means arranged to transmit signals to 

a computer in response to said position signals, 
2S orientation detecting means for detecting 

orientation of said sphere, and 
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processing mean3 configured to combine signals from 
aaiid touch eensing meana and paid oriencation detecting 
cnaa^a for suhae<3uenfc identification o£ a touch event 
orientation substantially with reapect to said graphic 
5 display means, 

X7 . Apparatuo according to claim IS, wherein said 
touich oensizag means comprises a plurality of microphones 
and said aphere has a surface that generates sound in 
3^0 reejponse to touch evcnto . 

XS. Apparatus according to claim X7, wherein said 
sphere has a roughened surface. 

19. Apparatus according to claim IS, wherein said 
computer includes processing meana configurable to 

perform steps of; 

selecting a graphical or text mode for said sensor, 
when, in said graphical mode to perform steps for 
ao identifying a graphical instruction in rcoponae to touch 

events , and 

when in said text mode to perform steps for 
identifying text in response to touch events - 

25 20, Apparatus according ro claim 16, wherein said 

sensor includes recharge^le cell means and a recharging 
inc^uctive loop means. 
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31 < A method of genearatins contarol signals for a 
eoinputer- generated environment arepr-esented gretph-ically 
on ia display apparatus, wherein a sensor having the form 
G of a fully enclosed sphere includes touch sensiiag rnean^ 

for generating position signals indicative of the 
position pf touch events on its sutrfacc, orientation 
sensing means for identifying the orientation of said 
sphere and transmitting means for transmitting sensor 
xo dat,-a. to a computer -generated ens^ironment , coinprising 

step£i of 

identifying a position of a touch event on the 
suMface of said sphere, 

identifying the orientation of said spher-e, and 
IS combining said position and orientation to generate 

couch event signals oriented substantially with respect 
to said display apparatus. 

22- .A method according Co claim 21, wherein said 
20 Couch sensing means includes acoustic transducer means ^ 

and said position signals are generated by processing 
eoiind signals. 

23. A method according to claim 22, wherein s^id 
25 sound processing includes frequency domain analysis. 
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24. A method according to claim 23, including 
i<ie;nti£ying pvobable di«ances £roctv censors o£ a touch 
e^r«^t in response to frequency characteristic. o£ .ound. 
at said eervaors . 

25. A method according to claim 24. wherein said 
distances are combined to identify a location, 

2S. A method according to claim 25, wherein 
additional processing of said signals Is performed in 
or^ear to reduCB; poaitional ei-ror. 

27. A method according to claim 23, including 
a,e^suxi«s a time interval between tlxe «tart of a sound 
at ;a plurality of acoustic transducer means. 

28. A method according to claim 27, wherein said 
sound is generated in response to a hit event on the 
surface of said sphere. 

29. A method according to claim 22, wherein 
frequency analysis is used to identify a characteristic 
of -drag touch evencs and start- time analysis is used to 
idenclfy a characteristic of hie couen evencs. 

30. A method of inceraccing wich a three 
dimensional object model in a computer generated 
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„iro™.ent, wh..eln .ign.ls supplied t= said 

a»vd.=nmez.t from a .urfac. t=u=h position sensitive 
sphere, comprising steps of: 

generating a mapping from said sphere to =-id 

objiect; and 

receiving toucH events from .»ia. spi^ore «»a 
interpret 3.ng tliem co interact 
portion of said object. 

31. A Tn«t.hod of modifying .a three dimensional 
ob^«ct n^oael ix. a computer generated environmenr, 
wherein signals are supplied said envircnmeni: from a 
..x;£a«. ro..K position sensitive sphere, comprising 

Steps of : 

generating a mapping from said sphere to said 
objtect ; 

receiving touch events from said sphere and 
interpreting them to manipulate a respective a-face 
portion of said object; 

updating said object; and 

generating a new mapping from said sphere to said 



2b 



3 2 . Apparatus e^afeet and. ally as 
wiqh reference Co Figures l, 2. 3 and 4. 



herein described 



Received 06-U-89 15:25 



from- To-THE PAT€NT OFFICE Page 42 



6 . DEC . 1999 16 : 42 ATKINSON & CO 

-pioi-Ga 



10 



40 



1- 1. -.-4. ^ ai 1 \» as herein described 

33. A method substantially as nere 

with reference to Figures 1, 2/ 3 and 4. 

34. A data processing sysrem for carrying out a 
method according to any one of claims 21 to 31. 

35. A computer-readable medium having computer- 
readable instructions executable by a computer such that. 
When executing said instructions, a computer will perform 
a method according to any one of claims 21 to 31. 
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An apparatus is disclosed £or supplying input 
signals to a coynputer. A sensor having the form of a 
s sphere has a touch sensi^±v^ surface for generating 

position data for touch events- The sensor includes 
oriprxtation sensors that determine rotation with respect 
to the earth's magnetic and gravitational fields, 
Oritentation data may be combined with position data to 

10 interpret the orientation of touch events on the surface 

wit>i respect to the computer's display. Cursor movement 
or text may be generated from -couch events. Preferably 
the sphere has q roughened surface that generates sound 
When touched- Position data is generated by proceasing 

IS sigpals from microphones under the sphere's surf ace - 

(Figure 1) 
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Figure 9 
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